ABSTRACT
PURPOSE: To assess the interobserver variability between chest radiologists in the interpretation of the Radiological Society of North America (RSNA) expert consensus statement reporting guidelines in patients with suspected coronavirus disease 2019 (COVID-19) pneumonia in a setting with limited reverse transcription polymerase chain reaction testing availability. METHODS: Chest computed tomography (CT) studies in 303 consecutive patients with suspected COVID-19 were reviewed by 3 fellowship-trained chest radiologists. Cases were assigned an impression of typical, indeterminate, atypical, or negative for COVID-19 pneumonia according to the RSNA expert consensus statement reporting guidelines, and interobserver analysis was performed. Objective CT features associated with COVID-19 pneumonia and distribution of findings were recorded. RESULTS: The Fleiss kappa for all observers was almost perfect for typical (0.815), atypical (0.806), and negative (0.962) COVID-19 appearances (P < .0001) and substantial (0.636) for indeterminate COVID-19 appearance (P < .0001). Using Cramer V analysis, there were very strong correlations between all radiologists' interpretations, statistically significant for all (typical, indeterminate, atypical, and negative) COVID-19 appearances (P < .001). Objective CT imaging findings were recorded in similar percentages of typical cases by all observers. CONCLUSION: The RSNA expert consensus statement on reporting chest CT findings related to COVID-19 demonstrates substantial to almost perfect interobserver agreement among chest radiologists in a relatively large cohort of patients with clinically suspected COVID-19. It therefore serves as a reliable reference framework for radiologists to accurately communicate their level of suspicion based on the presence of evidence-based objective findings.